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A DEVICE FOR MONITORING THE ENVIRONMENT 
OF A VEHICLE BEING PARKED 


Background Information 

The present invention is based on a device for monitoring the 
environment of a vehicle being parked, according to the 
species of the main claim. A device of this type is already- 
known from German Patent 43 3 6 28 8 CI, in which, for expanding 
the vision field of a camera, means are provided for swiveling 
the camera . 

Advantages of the Invention 

In contrast, the device according to the present invention 
having the characterizing features of the main claim has the 
advantage that, on the basis of a simple, robust arrangement, 
reliable monitoring of the vehicle environment can be assured. 
Movable parts, which are therefore subject to wear, are 
avoided, and the dead angles of the video camera are included 
in the monitoring area, so that the driver receives a danger 
message also in response to, for example, persons situated in 
the dead angle of the camera, and not only when questionable 
obstacles are located in the vision field of the camera. 
Therefore, there is an economic distribution of functions in 
detecting objects and reporting to the driver, between the 
object-detection sensors, on the one hand, and the camera, on 
the other hand . 

As a result of the measures indicated in the dependent claims, 
advantageous refinements and improvements of the device 
indicated in the main claim are possible. 

It is advantageous that the existence of objects outside of 
the vision field of the camera can be represented separately 
via a display unit. As a result, the attention of the driver 
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is immediately guided in the direction in which the danger 
threatens, although he is still unable to detect anything 
through the video image . 

5 It is particularly advantageous to provide an object-detection 
unit, which, for evaluating objects, processes both data from 
the object-detection sensors as well as the video images. 
Combining a plurality of sensor information units in an 
overall display has the advantage that the behavior of the 
10 system is always transparent because the driver can also check 
the information visually. The driver is not limited to the few 
parameters that isolated object-detection sensors would 
supply, such as the vehicle distance to the person behind- As 
a result, it is possible at any time for the driver, in a 
simple fashion, to carry out monitoring and driving 
v5 correction. 

■ SB! 

Q The visual presentation of the results in the image of the 

'L, recording video camera has the advantage that the driver can 

2XS very effectively survey the situation, since he is accustomed 

;~ to steer the vehicle based on visual information. 

Further advantages are derived from the features cited in the 
further dependent claims and in the description. 

25 

Drawing 

Exemplary embodiments of the present invention are depicted in 
the drawing and are explained in greater detail in the 
3 0 following description. The contents are as follows: 

Figure 1 depicts a top view of a vehicle having a device for 
monitoring, 

35 Figure 2 depicts a vehicle having a further device, 


Figure 3 
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depicts a schematic representation of a further 

2 


device, 

Figure depicts a video display unit 55, 
5 Figure 5 depicts a further video display unit 55 ' , and 
Figure 6 depicts a parking scene. 
Description of the Exemplary Embodiments 

10 

Figure 1 depicts a top view of a vehicle 7. In the rear space 
of vehicle 7, a video camera la disposed, which monitors the 
rear area of the vehicle in a vision field 8a. Mounted 
adjacent to video camera la are object-detection sensors 9a 
and 9b, which are used to detect objects in environment 

'"'■4 sectors 10a and 10b, respectively. These environment sensors 
p 10a and 10b directly adjoin vision field 8a, but they lie 

'O outside of it. In addition, forward-directed object-detection 

i=3 sensors 9c, 9d, and 9e are arranged in the area of the front 
2=9 bumpers of the motor vehicle, the sensors monitoring 

jg environment sectors 10c, lOd, and lOe, which are directed 

p toward the front area. 

Video camera la, non-movable and fixedly installed at minimal 
25 expense, covers a prescribed vision field 8a, which can be 

displayed to the driver by a video display unit arranged, for 
example, in the dashboard or the center console of the 
vehicle. Using object-detection sensors 9a and 9b, it is 
possible especially to detect objects that are approaching the 
3 0 area monitored by the camera but that are not yet registered 
in the image. Detected objects of this kind can then be 
reported to the driver even though they cannot yet be detected 
in the camera image. This can be, for example, a child that is 
approaching the vehicle from the side, although the driver 
35 cannot see this child either in the displayed image of the 
camera or in the rear view mirrors, because the child is 
located in the dead angle. The object-detection sensors are 
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configured, for example, as ultrasound, radar, video, or lidar 
sensors- Additional sensors 9c, 9d, and 9e can be used during 
parking maneuvers for measuring at all times the distance to 
the vehicle parked ahead or for detecting other (endangered) 
objects. If the distance to the forward vehicle or another 
object is too small, a message can be transmitted. 

However, the additional sensors can also be installed at any 
location in the vehicle to monitor the corresponding 
environment. For example, the time-related vehicle area offers 
further installation possibilities, in which objects can be 
observed and appropriate warnings can be provided to the 
driver. If the object detection is carried out in a monocular 
fashion, economies can be achieved because only one camera is 
used. However, such methods have the disadvantage, in 
comparison to stereo methods, that they do not deliver very 
precise and dependable results. Precisely by combining 
monocular object detection with other sensors such as 
ultrasound sensors, precision and reliability can be 
significantly increased . 

Figure 2 depicts vehicle 7, which has additionally installed 
video cameras lb and lc, whose corresponding vision fields 8b 
and 8c cover the front area of the vehicle. In this context, 
the cameras are mounted between obj ect -detection sensors 9c, 
9d, and 9e, so that the three aforementioned object-detection 
sensors, as a supplement to the cameras, monitor the 
peripheral areas of vision fields 8b and 8c. Furthermore, 
other obj ect -detection sensors 9f and 9g are mounted in the 
rear part of the vehicle, their corresponding environment 
sectors lOf and lOg partially overlapping vision field 8a of 
the video camera la and therefore, as a supplement to the 
video camera, bringing these areas of vision field 8a into the 
monitoring area. 

Providing additional obj ect -detection sensors in the rear part 
of the vehicle results in increasing the reliability of object 
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detection by generating data redundancy. If vehicle 7 is 
operated using a trailer, then, in the event that the trailer 
obscures the vision field or the environment sectors of the 
rear camera or of the rear object-detection sensors, the 
5 trailer can be provided with an analogous monitoring device 
la, 9a, 9b, 9f, 9g, which is directed at the area behind the 
trailer. Thus vehicles in trailer operation can also support 
the driver during parking maneuvers. 

10 Figure 3 depicts the schematic design of a device for 

monitoring the environment of a vehicle being parked, in which 
the totality of the video cameras is designated by reference 
numeral 1 and the totality of the obj ect -detection sensors is 

i.p designated by reference numeral 9. The video cameras and the 

l a | object-detection sensors are connected to a control unit 20. 

\J Control unit 2 0 has an image processing unit 2, which 

processes the image data of video cameras 1. Image processing 

□ unit 2 delivers preprocessed image data to an object-detection 
unit 30, which processes both the preprocessed image data of 

28 the video cameras as well as the signals of object-detection 
\j% sensors 9. Via an acoustical display 6, detected objects or 

□ danger situations can be reported to the driver. The 
acoustical display is arranged in the passenger area of the 
vehicle. A superimposition unit 40 is connected to image 

25 processing unit 2 and to object-detection unit 30. This 

superimposition unit 40 superimposes information supplied by 
object-detection unit 30 with respect to detected objects and 
the pre-processed image data from image processing unit 2 for 
representation in a video display unit 55 or 55'. Object- 

30 detection unit 3 0 is also connected to a maneuver calculating 
unit 50, which, from the object data of object-detection unit 
3 0 and externally supplied parameters 4, for example the 
selected steering wheel angle, calculates driving maneuvers 
and transmits this data to superimposition unit 40 for visual 

35 representation in the video display unit. Maneuver calculating 
unit 50 is also connected to a control unit 90 for the purpose 
of independently carrying out a driving maneuver. 
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In control unit 20, the images of the video cameras can be 
pre-processed using image processing algorithms, and they can 
be displayed on a video display unit 55 or 55 1 . In this 
context, the algorithms of the control unit can also fall back 
5 on vehicle parameters such as speed and the steering angle of 
the steering wheel- In the video display unit, in addition to 
the image content of the cameras, supplemental information 
such as warnings about objects in the vehicle environment can 
also be displayed- The possibility also exists of generating 

10 warnings acoustically via acoustical display 6. In this 

context, image processing unit 2 includes algorithms for image 
preprocessing, such as noise suppression, image rectification, 

m or the like. The processed images would be combined with 

uy supplemental image contents using superimposition unit 4 0 and 

i ; |f would be displayed on the video display unit. Ob j ect -detection 

! SS unit 30 receives data from the obj ect -detection sensors and 
? from the image processing unit. Known objects are transmitted 

! =3 to superimposition unit 40 for display in the video display 

unit and are also relayed to maneuver calculating unit 5 0 for 

2$ calculating maneuvers. In order to calculate maneuvers, 

external parameters can be taken into consideration. The 
p maneuver calculating unit can prepare the calculated maneuvers 
suitably for representation in the video display unit and, if 
appropriate, it can intervene in the control of the vehicle 

25 through a control unit 90. As examples of actuator systems, 

mention should be made of influencing the steering angle and 
intervening in the engine and brake control systems. Obj ect - 
detection unit 30, in monitoring in the environment, does not 
initially presuppose a preestablished parking geometry or the 

30 like, but rather, on the basis of the actually existing image 
or object-detection data, generates a description of the 
environment . The modeling of the environment and the images of 
the cameras are thus combined into one representation by 
superimposition unit 40. This representation is used to inform 

35 the driver comprehensively about the current situation in the 
vehicle environment. In this context, the object detection 
system supplies the location and the number of objects and, 
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depending on the sensor system used, it can supply varying 
object sizes in varying degrees of precision. This data (size 
and distance of the objects) can also be displayed in the 
video display unit, by object-detection unit 30 also 

5 transmitting this data to superimposition unit 4 0 in the 

appropriate manner. Using maneuver calculating unit 50, the 
present device, in addition to passively monitoring the 
momentary situation in the vehicle environment, can also 
actively assist the driver in controlling the vehicle. Object- 
10 detection unit 30 transmits to maneuver calculating unit 50 
the modeling data or the object-detection data of the 
environment. For certain scenarios, the vehicle course is then 
calculated by maneuver calculating unit 50. In what follows, a 

u3 few advantageous possibilities are indicated: 
i ; M 1. The vehicle course yields to detected obstacles. 

"*4 2 . The vehicle course leads into a parking space parallel to 
? the roadway . 

3 . The vehicle course leads into a parking space perpendicular 

i;3 to the roadway. 

5-;§ 4 . The vehicle course leads into a parking space diagonal to 
u g the roadway . 

5 . The vehicle course leads to a predetermined setpoint 
position between a plurality of obstacles, it being possible, 
e.g., to configure the latter. Thus an example of a setpoint 

25 position is the position in the home garage, and an example of 
an obstacle is the gate in front of this garage. For 
calculating the above-mentioned vehicle courses, it can also 
be taken into account that a trailer is attached to a vehicle 
and that the driver should initially countersteer in order to 

30 arrive at a predetermined setpoint position. Either the 
maneuver calculating unit is configured so that it 
automatically detects the above-mentioned different 
situations, or the driver has the possibility of selecting the 
corresponding parking variant in the selection means installed 

35 in the dashboard. Certain maneuvers, such as parking in the 

home garage or other standard maneuvers, can also be stored in 
the memory or be preprogrammed. For this purpose, the maneuver 
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calculating unit has a suitable program memory, from which the 
stored maneuvers can be retrieved. 

From the above-mentioned considerations, the following 
5 gradations are yielded for the degree of vehicle intervention 
on the part of the device according to the present invention: 
1. The images of the video cameras are displayed in the video 
display unit mounted in the dashboard or in the center 
console . 

10 2 . At the same time, object information, such as size, 
position, and distance, are overlaid in the appropriate 
manner * 

:»» 5 3. Additionally, information regarding the vehicle condition 
';Q is overlaid, such as the steering angle adopted, which 
15 determines the appropriate driving path, the vision angle of 
.^-f the vehicle with respect to the road (i.e., the angle of the 
:; p normal through the automobile relative to the road normal) , 
lxtS etc. 

□ 4 . The steering maneuvers are calculated by maneuver 

?B calculating unit 50 and are displayed in the video display 
unit. The steering angle is calculated by the system as a 

rf function of the current situation and is overlaid on the 
camera image in addition to the current vehicle steering 
angle. On the basis of the overlaid steering angle, the driver 

25 makes a variance comparison and turns the steering wheel 

accordingly. In this manner, the driver continually maintains 
complete control. The necessary steering direction can also 
(additionally) be indicated by correspondingly overlaid 
arrows . 

30 5. The device automatically sets the calculated steering angle 
via control unit 90, so that there is a direct intervention in 
the steering. However, the steering angle can at any time be 
overridden by the driver, and thus an individually appropriate 
steering angle can be chosen. The driver himself thereafter 

35 controls the vehicle longitudinal motion, i.e., the engine and 
brakes . 

6. In a completely automatic operating mode, the device 
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controls the vehicle entirely automatically by intervening 
both in the steering as well as in the engine control on the 
basis of the calculated vehicle maneuvering data. Here too, 
the maneuver can at any time be broken off by the driver. 

5 

Object-detection unit 30 and maneuver calculating unit 50 in 
an alternative embodiment can be configured so that the active 
maneuver control is automatically broken off in accordance 
with Point 6, for example, in the following situations: 
10 1. Objects, especially rapidly moving objects, appear in the 
driving area, especially in the rear vehicle area. 
2. Objects were surveyed in a grossly mistaken fashion. 

,==i 3. There exists an acute danger of collision with an object. 

-3 4. Endangered objects (living things) appear. 

ii 

Figure 4 depicts an embodiment 55 of the video display unit 

:: p for displaying the images supplied by the cameras and 

u=j processed by the control unit. Video display unit 55 has an 

Q image area 12 as well as both vertical edge areas 13 as well 

2|S as horizontal edge areas 14, which together form the border of 

up image area 12 . 

In image area 12 itself, and in the vertical as well as the 
horizontal edge areas of the image area, colored bars are 

25 displayed for optically signaling objects in the vehicle 

environment that is not covered by the vision field of video 
camera (S) . Additionally, at a suitable location, a pictogram 
15 can be overlaid for representing the vehicle itself. In the 
environment of this vehicle, the position of detected objects 

3 0 can also be plotted. Video display unit 55 can advantageously 
be integrated in the dashboard or in the center console of the 
vehicle . 

As the warning color, a signal color such as red can be used 
35 in the edge areas. Depending on which bar lights up, the 
driver knows immediately to which side of the vehicle he 
should direct his attention, although the camera image still 
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is not supplying any information to him. Additional support 
can be provided, parallel to this, by acoustical display 6. 

Figure 5 depicts a further embodiment 55' of the video display 
5 unit. In image area 12, a cube 115 is depicted, which in its 
size and shape roughly represents a detected object. In area 
116, data can be overlaid for the distance or the size of this 
object. Lines 16 mark the driving path of the vehicle, if the 
given steering angle were to be maintained. Lines 17 mark the 
10 driving path which the vehicle would cover if the driver were 
to follow the calculated route. Arrows 18, which light up, 
alternatively, depending on the direction in which the driver 
.™ is supposed to turn the steering wheel, show him how he should 
5 i0 steer in order to achieve the steering angle suggestion that 
lM is indicated by lines 17. 

= E All of the cited information is superimposed in image area 12 
along with the image of the video camera in order to provide 

g rapid and precise information to the driver concerning the 
2|;§ situation. When the vehicle guidance is performed manually, it 

:,q is also possible, if sensors and cameras are also present in 

H the forward vehicle area, to provide an automatic switchover 

between the front and the rear video camera depending on which 
gear the driver has currently selected. 

25 

Figure 6, by way of example, shows a parking maneuver of 
vehicle 7 behind parked vehicle 107 and in front of parked 
vehicle 108. The same reference numerals as in the previous 
Figures designate the same parts and are not described once 
3 0 again. 

In particular, laterally arranged object-detection sensors 9a 
and 9b monitor areas that cannot be observed by fixedly- 
installed camera 8a, but which are relevant for parking, 
35 especially if careless pedestrians, for example, enter into 
the areas just outside vision field 8a of camera la. In this 
case, the driver receives through the video display unit an 
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appropriate warning and can act accordingly until these 
pedestrians once again move out of the dead angle of the 
vehicle. Front object-detection sensors 9c, 9d, and 9e also 
supply the precise distance to forward vehicle 107 and make 
parking easier for the driver especially in the case of 
vehicle chassis that are not completely visible, making the 
borders of the vehicle hard to determine from the driver's 
seat . 
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